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Abstract 

In many resource allocation problems, a centralized controller needs to award some resource 
to a user selected from a collection of distributed users with the goal of maximizing the utility 
the user would receive from the resource. This can be modeled as the controller computing an 
extremum of the distributed users’ utilities. The overhead rate necessary to enable the controller 
to reproduce the users’ local state can be prohibitively high. An approach to reduce this overhead 
is interactive communication wherein rate savings are achieved by tolerating an increase in delay. 

In this paper, we consider the design of a simple achievable scheme based on successive refinements 
of scalar quantization at each user. The optimal quantization policy is computed via a dynamic 
program and we demonstrate that tolerating a small increase in delay can yield significant rate 
savings. We then consider two simpler quantization policies to investigate the scaling properties 
of the rate-delay trade-offs. Using a combination of these simpler policies, the performance of the 
optimal policy can be closely approximated with lower computational costs. 

Index Terms 

Quantization, interactive communication, resource allocation, dynamic programming 

I. Introduction 

A common pattern in resource-limited systems is the allocation of the resource by a controller 
among a set of competing consumers/users. In an effort to make the most efficient usage of the 
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resource, the controller awards the resource to the user that derives the most utility from it. This 
can be modeled as the controller computing an extremum (max, min, arg max, arg min) of the 
distributed users’ utilities. By computing the arg max, the controller can allocate the resource to 
an appropriate user without knowing exactly its value. Computing the max of the users’ local state 
allows the controller to make a global decision, e.g., turning on building-wide air conditioning based 
on the hottest room. Computing both the max and the arg max across users would provide the 
controller with knowledge of both the user to award the resource to, and the utility this best user 
would receive from it. 

Often, these users are not colocated and must communicate their utility to the centralized con¬ 
troller. This communication presents an overhead and techniques for minimizing the required rate 
are needed. In some cases, the rate can be reduced by incurring only a small penalty. Rate-distortion 
theory is an example where the rate required to communicate/reconstruct a signal can be reduced 
if small errors in the reconstruction are tolerated. Interactive communication allows an alternative 
framework wherein communicating parties can send messages back-and-forth over multiple rounds 
[1]. This back-and-forth messaging can reduce the rate required to compute an extremum/extrema 
of the sources at the cost of increased delay. In the rate-distortion framework, an achievable scheme 
for a collection of distributed users talking to central controller (multiterminal CEO) is quantization 
followed by entropy (i.e., Huffman) encoders [2], [3]. In this paper, we consider this same technique 
in the context of interactive communication. We formulate the design of the multi-round quanti¬ 
zation as the solution to a dynamic program and demonstrate a substantial reduction in overhead 
communication rate can be obtained for a small increase in delay. The use of dynamic programming 
to solve for optimal quantization policies in the interactive multiterminal source coding problem 
parallels earlier work that used dynamic programming to solve for optimal quantization of a single 
source [4]. Because of the high computational complexity of the dynamic programming problem, 
we identify two simpler schemes and investigate their performance over a range of parameters. By 
combining these two schemes, we demonstrate a close approximation of the dynamic programming 
solution at a lower computational cost. 

The paper is organized as follows. We review related literature from information theory, signal 
processing, and communication complexity in §11. We present the basic problem model and establish 
the mathematical notation used throughout the paper in §III. In §IV, we formulate the optimal 
rate-delay trade-off of scalar quantization as the solution of a minimum cost dynamic program. We 
characterize the set of terminating states for the dynamic program when computing the different 
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extremum functions under consideration (cf. Prop. 1 & Prop. 2). We prove that, asymptotically 
in the number of users, the cost for the CEO to compute the different extremum functions are 
equal (cf. Prop. 4). In §V, we restrict our attention to the case of users’ utilities being distributed 
uniformly. This assumption allows us to provide analytical expressions for the rate and delay of simple 
quantization policies (cf. Prop. 5 & Prop. 6). We then provide an extension to these simple schemes, 
which is asymptotically (in the number of users) sufficient for minimizing the cost of computing the 
selected extremum functions (cf. Prop. 8). The proposed family of quantizers is signihcantly smaller 
compared to the space of all possible quantizers. Hence, with these results the dynamic program 
can be solved more quickly with a relatively small incurred penalty relative to the optimal dynamic 
program. In §VI, we show the rate-delay trade-offs for different distributions and varying numbers 
of users. We also present a comparison of the minimum cost from searching over all quantizers to 
the minimal cost from searching over our proposed family of quantizers. We present directions for 
future work in §VII. 


II. Related Work 

In the present work, we consider a distributed model where all users communicate to a central 
node that wishes to compute a function of the users’ sources. In particular, the users cannot overhear 
each other but can design their codes (with knowledge of the source distributions of other users) to 
communicate with the central node—called “cooperative design, separate encoding” [5]. These models 
are referred to as the chief estimation officer (CEO) problem [6]. We begin by reviewing fundamental 
limits and achievable schemes for non-interactive variants of CEO-type problems. We then review 
results for interactive variants that demonstrate signihcant rate savings may be possible. The non¬ 
interactive CEO problem has received considerably more attention than the interactive variant, and, 
for the cases where fundamental limits are known, quantization followed by entropy coding closely 
approximates these limits. This motivates our study of interactive quantization as a means to realize 
further rate savings at the expense of an incurred delay. 

A. Non-interactive communication: fundamental limits 

In information theory, the interest is usually on characterizing inner/outer bounds for the rate 
region. Berger et al. introduced the generic CEO problem, wherein the CEO wants to reproduce the 
source from the received signals [6]. The rate region for the problem of source reproduction with 
constrained distortion remains unknown, except for the cases of: i) jointly Gaussian sources with 
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quadratic distortion [7]-[9]; ii) finite alphabet sources with logarithmic loss [10]; or, iii) discrete source 
distributions which are independent across users [11]. Vempaty and Varshney considered the CEO 
problem for non-regular source distributions (e.g., truncated Gaussian) with quadratic distortion 
and studied the asymptotic behavior of the distortion function [12], 

In our work, we are interested in having the CEO compute a function of all sources; this is referred 
to as the distributed function computation (DEC) problem, and was considered in [13]-[15]. This 
general formulation contains the specialized problem of function computation with side information; 
in this problem, the CEO knows all but one of the sources [16], [17]. When the problem requires 
error-free computation, it was shown that the minimum worst-case rate is related to the chromatic 
number of the characteristic graph of the source [16]. In the case of lossless (in the Shannon sense) 
computation, it was shown that the minimum average rate is the conditional graph entropy of the 
characteristic graph of the source [17], [18]. Building upon this line of research, the rate region for 
the lossless DEC problem was characterized for certain problem instances [14], [15]. Sehdgaran et 
al. derived inner and outer bounds to the rate region for a class of tree-structured networks (which 
includes the CEO problem) and showed that the inner and outer bound coincide with each other 
if the sources obey certain Markov properties [14]. Doshi et al. gave the rate region for the DEC 
problem under a different constraint that they referred to as the “zig-zag” condition [15]. They 
showed that any achievable rate point can be realized by graph coloring at each user and Slepian- 
Wolf (SW) [19] encoding the colors. Han and Kobayashi partitioned all DEC problems based on 
whether their achievable rate region coincides with the SW region [13]. 

The aforementioned literature provides insightful outer bounds for comparing the performance 
of distributed quantizer designs [20], [21], but the achievable schemes used in the proofs usually 
require block coding with inhnite block length, which is not practical. For use in a real system, 
simpler achievable schemes with low computational complexity and performance close to the limits 
are needed. 

B. Non-interactive communication: achievability 

A concern of signal processing is to provide optimized practical quantization algorithms for the 
DEC system with performance close to the rate-distortion limits [2], [3], [5], [22]-[25]. There are 
asymptotic results for sufficiently high-rate and low-distortion that are derived by applying high rate 
quantization theory [26], while there are also non-asymptotic results derived from generalizations of 
Lloyd’s algorithm. 
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For the high-rate and low-distortion scenario, Misra et al. considered a qnantization scheme for 
the analysis of distribnted scalar qnantization [23]. It was shown that, with certain constraints on the 
objective fnnction and sonrce distribntions, the high-resolntion approach can asymptotically achieve 
the rate-distortion limits, and the optimized qnantization is regnlar^. Snn et al. nsed a similar high- 
resolntion approach, bnt with a simpler decoder design and relaxed sonrce distribntion reqnirements 
[24]. 

For the general rate-distortion problem, an algorithm for bnilding optimized distribnted qnantizers 
was given wherein the CEO nses the qnantized observations to perform hypothesis testing [5]. A 
two-stage distribnted scheme was proposed for the case when the nsers each have a noisy observation 
on the same sonrce, and the CEO needs to reprodnce the sonrce with a bonnded expected distortion 
[2], [3]. A hrst stage of local qnantization is followed by a second stage of encoding the qnantized 
signals based on Slepian-Wolf coding nsing syndrome codes [3] or index rense techniqnes [2]. In 
their treatment of the CEO problem with non-regnlar sonrce distribntions, Vempaty and Varshney 
provided an acheivability proof that ntilized a layered approach of qnantization followed by entropy 
coding [12]. 

Most of the provided distribnted qnantization schemes are non-interactive, which means the nsers 
each commnnicate with the CEO once, and no feedback is allowed from the CEO. For the problems in 
which the non-interactive fnndamental limits are known, distribnted qnantization shows satisfactory 
performance as compared with the limit. However, very little work has been done in the interactive 
distribnted cases. 

C. Interactive communication 

Interactive communication is a scheme that allows message passing over mnltiple ronnds. At each 
ronnd, the commnnicating parties are allowed to send messages based on what they have received in 
previons ronnds as well as their local sonrce observation [1]. The interactive commnnication literatnre 
is ronghly divided into two categories: commnnication complexity and interactive information theory. 

The commnnication complexity literatnre is concerned with Ending commnnication protocols 
that minimize the snm-rate snbject to different sets of constraints. Overviews of commnnication 
complexity can be fonnd in [27], [28]. Commnnication complexity is defined as the snm-rate cost 

quantizer is called regular if each partition cell is an interval and each output level is within the corresponding 
interval. 
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minimized over all protocols and maximized over all possible input pairs (worst-case cost). Average 
cost has also been studied for randomized coding protocols. Much of the communication complexity 
literature is focused on 2 users. Models with an A-terminal setup were considered in [29], [30], where 
the authors focused on providing communication complexity bounds with the restrictions that the 
function must be Boolean and the message sent at each round must be binary. However, in our work, 
we are interested in providing achievable schemes for problems without the limitation to 1 bit of 
communication in each round or the restriction to Boolean functions. 

Kaspi determined the two party information theoretic limit for lossy compression via interactive 
communication [1]. This line of research was continued by Ma and Ishwar, who showed (by an 
example) that the minimum rate for a given distortion constraint can be arbitrarily smaller than the 
non-interactive minimum rate obtaining the same distortion [31]. In follow up work, Ma and Ishwar 
showed (by an example) that for the DFC problem, the minimum sum-rate for losslessly computing 
a function can be smaller than the non-interactive rate; even inhnitely-many rounds of interaction 
may still improve the rate-region [32]. 

These results motivate us to consider interaction for the DFC problem. In earlier work, we 
considered the non-interactive DFC problem of computing an extremum of independent users. We 
developed distributed scalar quantizers with rate-distortion performance close to the rate-distortion 
limits [20], [21]. We provided an achievable interactive communication scheme where the CEO 
communicates a threshold to the users at each round and the users reply with a single bit indicating 
if its value is above or below the threshold [21], [33]. This scheme can be thought of as a simple 
two-bin quantizer selected by the CEO at the beginning of each round; in the present work we extend 
this by allowing the CEO to select a multi-bin quantizer in each round. 

This interactive coding scheme can be understood as a type of posterior matching [34]. Shayevitz 
and Feder considered the problem of point-to-point communication over a memoryless channel with 
noiseless feedback from the receiver. A capacity-achieving transmission scheme was developed based 
on the transmitter providing statistically independent information that is missing by observing the 
a-posteriori density of the message as feedback from the receiver. In the present work, the focus is 
on minimizing the sum-rate from a collection of sources; however, the feedback from the CEO is 
used by the users in determining what is transmitted in the next round. 
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Z = arg max Xi or max Xi 


Fig. 1. Interactive quantization system diagram. The users’ utilities Xi are quantized using the quantization function 
giving the quantized utilities which are then entropy encoded (Enc) before being sent to the CEO (Dec). 
Based on the received quantized utilities, the CEO updates the quantization function at the users until the desired 
function has been computed. 


III. Problem Model 

We assume that we have N users, each with local state Xi for i = 1,... ,N, vying for a resource 
to be allocated by the controller (i.e., CEO). We model the users’ local states as i.i.d. (across users) 
discrete random variables with support set X and PMF px{x), and interpret state as a proxy for 
the users’ utility. Maximum utility of the resource is obtained by allocating it to the user with the 
largest local state value. Without loss of generality, we will take X = {1,... ,L} where L is the size 
of support set. To select a user to award the resource to, the CEO wishes to compute one of the 
following functions^: 1) argmax^W; 2) max^ W, or; 3) (argmaXj W, max^ W). 

We view quantization as a function that maps the hnite support set df C R onto another hnite set 
df, i.e., <5 : df — 7- df. Traditionally, quantization is used as a lossy compression scheme for representing 
sources with values drawn from a continuous support set. In this case, the quantizer is specihed by 
partitioning the support set into intervals that are mapped to representative values. Implicit is the 
assumption that the quantizer is monotonically increasing x,y £ X s.t. x < y Q(x) < Q(y). 

When X is hnite (as we assume in this work), the quantizer can still be specihed in terms of intervals 
but this representation may not be unique. With the assumptions of X hnite and Q order preserving, 
we can alternately specify a ilT-level quantizer as a il'-tuple of integers n = rti,..., nx that sum to L 
with the following interpretation: the hrst ni elements of X ({1,..., ni}) are mapped to 1, the next 

■^Though we focus exclusively on the case of maximization in this paper, similar results hold for the case of 
minimization. In later sections, we enforce a decreasing order on certain parameters; to obtain results for minimization, 
the order should be increasing. 
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n 2 elements of X ({ni + 1,... , 71 - 2 }) are mapped to 2, and so on. This representation is unique (in 
that different integer tuples correspond to different quantization functions) and the set of possible 
scalar quantizers is isomorphic to the compositions of the integer L (cf. Dehnition 1 in §IV), which 
are the sequences of positive integers that sum to L [35]. For brevity, we dehne ni-k = ni + • • • + nfc. 
The induced PMF on Q is then 

'^l:k 

PQ(.k) = ( 1 ) 

We assume that time is slotted into rounds of sufficient length that the CEO can communicate to 
the users and receive their responses in a single slot. We indicate the time slot/round of interaction 
by t. In our analysis, we assume that feedback from the CEO provides all users with the same 
knowledge as the CEO. 

Suppose that at the beginning of the t-th round of interaction, the CEO observes that there are 
active users and the support set has size The CEO will select a quantization function 
(homogeneous across users) by selecting the number of quantization levels and the quantization 
bin sizes = (nf\... and communicate this to the users. Let = Q^^\Xi) denote the 

quantization bin in which W lies and let = {Qf'^ : i = 1, ■ ■ ■, denote the length tuple 

of response received by the CEO from the users. Dehne 

Ar(t) 

= (2) 

i=l 

as the number of sources with state in bin k and let A;/ = maxjA; : > 0} = max^ be the 

largest of the indices of non-empty quantization bins. Based on the responses from the active 
users, the CEO performs the following updates in each round of interaction: 


Arh+i) ^ 


^(i+i) 


= n 


(t) 


(3a) 




(i) = 


Pq\K) 


Q^\i) = kt 


(3b) 


1^0 o.w. 

The hrst equation captures the fact that if a user’s quantized value is not in the highest reported bin, 
the user’s utility cannot be a maximizing value. Only the users with values in the highest reported 
bin need to continue interacting with the CEO; the remaining users become inactive and do not 
participate in subsequent rounds. The second equation updates the cardinality of the support set; 
the maximizing value W corresponds to one of the values of X^^'^ that maps to the maximum reported 
quantization bin. The hnal equation updates the PMF for the remaining range of user values. At this 
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point, the CEO is ready to begin the t+ 1-th ronnd of interaction. Since the initial alphabet of nser 
observations is hnite and the size of the alphabet decreases in each ronnd (3a), interaction is 
gnaranteed to terminate after at most ronnds. In the following, we will omit the time snperscript 
when the time instance t is not relevant to the discnssion and/or is clear from context. 


IV. Optimal Solution via Dynamic Programming 
At time t, the CEO observes the state of the system and wishes to compnte 


a qnantization policy G that minimizes the cost of compnting the desired 

fnnction /: 


Cj (.'•>) 


= mm 


(1 - \)R (aW, + At + E [C/ 


A) 


( 4 ) 


The hrst part of the term inside the minimization consists of a weighting (A) of the rate and 
delay incnrred by choosing the qnantizer given by when in state The parameter A is hxed 
thronghont and sets the relative importance of minimizing the rate {R{-)) versns the delay (t(-))- 
Given s*-*) and the rate and delay are: 


0 


o.w. 




1 sW ^ 5 

0 o.w 


/ 


(5a) 


(5b) 


where Pq^ is given by (1) and the set S* represents terminating states (from which / may be 
compnted). In general, the set S* will depend on the particnlar extremnm fnnction being considered. 
If ^ S*, then the CEO will select a qnantizer and the rate incnrred is the entropy of the indnced 
PMF on the qnantized valnes times the nnmber of active nsers ; the delay is an additional ronnd 
of interaction. If G S*, the CEO can compnte / and the interaction is over. 

The second part of the expression inside of the minimization consists of the “cost to go”. Depending 
on the particnlar state and particular action chosen, the system state transitions to state 
sh+i) with some probability The “cost to go” is an expectation of the optimal cost function taken 
over all reachable next time step states given the current state and is given by 


E 


Q (S<‘+ 


1 ) 




A(*) 

= E 

k=l 


Ar(t) 


Y,p{k,i)Cf [{i,n^k\Px^^^) 


i=l 


( 6 ) 
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where 

( k-l \ ^ ‘ 

'^PqU)] j- (7) 

The outer summation conditions on the largest reported quantization bin {k^ = k) while the inner 
summation conditions on there being i users in the k^-th bin. Given these two outcomes, the state 
in the next time step is given as 

The next result characterizes the set S* of terminating states for the functions argmax, max, or 
the pair (argmax,max). 

Proposition 1. When the CEO wishes to determine the argmax of the set of users’ values, G 5)^ 
iff = 1 or = 1. 


Proof: If at some time t there is only one user still contending for the resource it must be the 
unique maximizer. If the set of possible values consists of a single value, then all remaining users’ 
values equal this value and they are all maximizers. In either case, the CEO has losslessly determined 
the set of argmax users. ■ 


Proposition 2. When the CEO wishes to determine either the max or the pair (argmax,max) of 
the set of users’ values, G iff = 1. 


Proof: If the set of possible values consists of a single value, then all remaining users’ values 
equal this value and they are all maximizers. ■ 

The set of terminating states is larger when the CEO wishes to determine the argmax because 
communication can stop when a single user is left, regardless of the set of remaining possible values. 
When determining the max, the CEO still needs subsequent rounds of communication with the single 
remaining user to determine its value. The following gives the optimal cost for G S\ \ S\^. 


Proposition 3. Eor G 5(4 \ the optimal quantization strategy is for the single remaining 
user to entropy eode the state, and thus eomplete the eommunieation in one additional round, i.e., 
(sW) = (1 - \)H (pW) + A. 

Proof: First, we show that for any quantizer n the expected rate is H (^p^x\ proceed by 
induction. The base case of = 2 is immediate. 

For the inductive step, suppose we have a quantizer n = (ni,..., nx)', then 


i?(sW) = EI(pW) + E 


W) 


( 8 ) 
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where 

kW 

H{Pq) = - (^)log2PQ (^) (9) 

k=l 

and with the inductive assumption it can be shown that 

if(*) l(*) 

E sW] = ^p{J^(A:)log2p[5^(A:) - log2P^\i). (10) 

k—1 i=l 

We conclude that R{L^^'>) = H the cost depends only on the delay. A minimum delay of 

1 is achieved by n = (1 ,..., 1). ■ 

The next proposition shows how the cost of computing the argmax is related to the cost of 
computing the max. 

Proposition 4. The cost for the CEO to compute the max exceeds the cost of computing the arg max, 
but the difference between the two costs goes to zero as the number of users N increases. We have 



where A —)■ 0 as N ^ oo. 

Proof: Recall that C S\ (Prop. 1 & Prop. 2). Therefore, an optimal quantization policy for 
computing the max is a feasible quantization policy for computing the arg max and the lower bound 
is immediate. 

To establish the upper bound, an optimal quantization policy for the argmax can be extended 
into a feasible quantization policy for the max. Starting at time t = 0, the CEO follows the optimal 
quantization policy for argmax until some time t = 5 such that G If G S^, then 
the CEO has determined the max. If G S\ \ then the CEO has determined who the 
unique maximizer is but not what their value is. Since there is a single user left, the minimum cost 
CO) = (1 ^ (^p^x^ + A (cf. Prop. 3). We have 

H (pf) < log 2 < log 2 - l) (12) 

where the last inequality follows from observing that the size of the support set decreases by at least 
one at each round and 5 > 1. As this is a feasible quantization policy for the max policy we have 

s('5)g5*\5^ (13) 

< 4*) + p (£:) [(1 - A) iog2 -1) + a 
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where £ is the event G 5^ \‘5^| and P is the probability of the qnantization policy 

ending in the state We have that £ C {jargmaXj = 1}. Snppose that there are two or more 
maximizers. Then, for any seqnence of qnantizers, the set of maximizers will respond with the same 
qnantized ntility and the only possible terminating state is G 
To show that A —?■ 0 as X —?■ oo it snffices to show that 


lim P 

N^OO 


arg max Xi 



(14) 


Let A denote the event that |argmaXjXi| = 1 and Bk denote the event max^X^ = k. By the law of 
total probability we have 

L L 

P (X) = ^ P (X n Sfc) = ^ NFx{k - l)^-^px{k). (15) 

k=l fc=l 


Since 


lim NF^-^ 

N^oo ^ 


{k - l)px{k) = 0 


VA:= 1,...,L, 


(16) 


it follows that P(X)—7-OasX—j-oo ■ 

To trace ont the rate-delay trade-offs of interactive scalar qnantization, the dynamic program of (4) 
is solved for mnltiple valnes of A and the corresponding rate-delay valnes are plotted parametrically. 
This is done in §VI for representative sonrce distribntions. The following dehnitions of two related 
combinatorial objects are needed before discnssing an assnmption concerning the search space 
in (4). 


Definition 1 (Compositions & partitions of an integer [35]). The compositions of an integer L are 
the seqnences of positive integers that snm to L. The partitions of an integer are the ways to write 
it as a snm of positive integers, disregarding order. 


For example, consider the integer 3: there are 4 compositions (namely {(3), (2,1), (1, 2), (1,1,1)}) 
while there are only 3 partitions ({(3), (2,1), (1,1,1)}). 

Assnmption 1. ITe take the set of quantizers in solving (4) to be the set of all partitions of the 
integer L instead of the set of all compositions. 


Remark. As a jnstihcation for the above assnmption, recall that in §III we showed that set of 
qnantizers is isomorphic to the set of compositions of the integer L. Using basic combinatorial 
argnments, it can be shown that the nnmber of compositions c{L) for a given integer L (and thns 
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Fig. 2. Comparison of compositions and partitions. Shown is the rate-delay trade-off obtained by optimizing over 
the set of compositions (markers) and partitions (no markers) for uniform sources with support set size L = 16. 


the number of possible quantizers) is 


c{L) = 2^-\ 


(17) 


G. H. Hardy and S. Ramanujan [35] obtained the following expression for the asymptotic behavior 
for the number of partitions (p{L) 


(p{L) ~ 


- 7 = exp 

alVs 



n —)■ oo. 


(18) 


For computational tractability, we take the set of quantizers to be the set of all partitions instead 
of compositions. Our justihcation for doing so is shown in Fig. 2. For a small initial number of 
users N and initial support set size L = 16, we computed the rate and delay cost components for 
every composition and every partition assuming subsequent rounds are solved optimally. The Pareto 
optimal boundary for compositions (markers) matches the boundary for partitions (no markers) for 
these parameters. 

For a given source distribution, as the number of users increases, the distribution of the maximum 
becomes more and more “peaked” about the largest possible value; this is made more rigorous in 
Lemma 1. Since the CEO is seeking to identify either the argmax or max (or both), we expect 
that quantizers that more hnely quantize the larger values of the support set will outperform those 
quantizers that do not. 


Lemma 1. Suppose Xi ~ px{x) where x takes values in the finite alphabet X = {xi,... ,xl}, then 
the sequence Yn = maxi<i<„Xi converges almost surely towards xl- 
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Proof: It is easily established that Pn(e) = P (|^n — xl\ > e) < P {¥„ < xl-i) for all e > 0 from 
which it follows 

_ 1 — T 

(19) 


2_^ Pn{e) < -^^ < oo 


Px{xl) 

and therefore converges almost snrely towards xl (cf. Theorem 7.2.4 in [36]). ■ 

In formnlating the cost fnnction of (4), only the rate on the nplink (i.e., nsers to CEO) was 
considered. The reason for focusing exclusively on the uplink costs are concerns of asymmetric 
power constraints when applying this framework to cellular systems. In this context, the users are 
the battery-constrained mobile stations while the CEO is the basestation. This motivates the next 
assumption concerning (4). 


Assumption 2. ITe assume the eost of eommunieation from users to the CEO is more expensive 
than eommunieation from the CEO to the users. Therefore we omit the eost of dissemination on the 
downlink from the CEO to the users in our analysis. 


Remark. As a justihcation for the above assumption, note that after collecting all the responses 
from the users, the CEO observes the next state and needs to convey this back to the users. 

Assuming the CEO broadcasts feedback to the users, the rate of feedback i?cEO bounded from above 
by 

Rc^o < H . (20) 


g{t+i) _ 


Observe that this entropy over-estimates the rate required: at the end of interaction, not only does 
the CEO know the value of the function, but so do all the users. Dehne 

git+i) 5-(t+i) H s* 

^ ( 21 ) 

0 ^ S) 

which condenses all of the terminating states for the function / into one. This allows the CEO to 
signal the end of interaction, without the extra rate to convey the function result back to users. 
To compute an upper bound on the optimal cost including the rate of feedback from the CEO, the 
objective function of (4) is changed to 




= mm 

a(t)g^(t) L 


(1 - A) {r + E [C/ -(22) 


Fig. 3 shows the impact on the rate-delay trade-offs of the optimal scalar quantization policy The 
effect is to increase the required rate for a given delay (or increase the required delay for a given 
rate), with the effect being most pronounced when the number of users N is small. These results 


DRAFT 


September 8, 2015 



15 


6 

5 

4 


■ii 3 


DC 


2 


1 


1 2 3 4 5 

Delay [no. of rounds] 


2 3 4 5 

Delay [no. of rounds] 


2 3 4 5 

Delay [no. of rounds] 



Fig. 3. Comparison of rate-delay trade-offs with and without including the cost of feedback from the CEO to the 
users. The source distribution was uniform with support set size L = 16. 


are for the case where the cost of communication from the users to the CEO equals the cost of 
communication from the CEO to the users. A more nuanced approach would modify (22) to include 
a weighting factor to adjust the relative importance of the rate from the users to the CEO versus 
the rate from the CEO to the users. Since the impact of including the rate of feedback is already 
marginal, this would only further reduce the differences in the rate-delay trade-off curves. 

V. Analysis of Suboptimal Schemes 

The dynamic programming formulation of the previous section is amenable to computing the 
minimum cost, and therefore the rate-delay trade-off, of interactive scalar quantization as an achiev¬ 
able scheme. A drawback with computing the solution to the dynamic program is that it does not 
provide insight into how the minimum cost scales in the number of users N and/or support set size L. 
Additionally, the computation provides little insight about the structure of the optimal quantizers. In 
this section, we consider two simple quantization strategies and derive expressions for the associated 
rate and delay. We then generalize to a family of strategies and prove their near-optimality; the 
signihcant reduction in the size of the search space results in faster computation of (4) with only a 
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small penalty. 

In this section, we assnme the nsers’ ntilities have a nniform distribntion. This assnmption is 
motivated by the resnlts of §VI-A and the analytical tractability of the resnlting rate and delay 
expressions. 


A. Binary search 


We hrst consider a qnantization strategy for compnting the argmax inspired by binary search. 
At each ronnd, the remaining snpport set is divided in half and the nsers indicate whether their 
valnes lie in the lower or npper half. This process is repeated nntil either a single nser remains or 
the snpport set has been rednced to one. 

We assnme L is a power of two to repeatedly divide in half. The rate for this scheme is given by 

Rh{N,L) + Y.t2 


Rb{N,2L) = N + 


2N 


(23) 


with base cases •) = Rb{-^ 1) = 0. Since the initial snpport set size (2L) is being halved, each 
nser replies with a single bit for a total of N bits. Depending on the nsers’ valnes, all N nsers conld 
be in the lower half {L) which happens with probability 2“^; or i nsers conld be in the npper half 
which happens with probability . Following a similar line of reasoning, delay is given by 

n{N,L) + j:l 2 On{t,L) 


2L) — 1 + 

with base cases rb(l, •) = rb(-, 1) = 0. 


2N 


(24) 


Proposition 5. If L is a power of two, the expected rate of computing the argmax with binary search 
is 


Rb{N,L) = 2N (l-^) 


(25) 


and the expected delay is bounded by 


Tb{N, L) < min{log 2 N + 1, loga L] 


(26) 


for N >2 and L >2. It follows that 


L^oo JV 


lim n{N^ L) < log 2 + 1. 

L^oo 


(27) 
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Proof: We prove the expression for expected rate by induction. The base case oi N = 2 and 
L = 2 is immediate. For the inductive step, we have 
Rb{N,2L) 


AT + 2-^ [2iV (^1 - ^ V 27 - i 


L 


(28) 


+ ( 1 -^) 

where (a) follows by the inductive assumption and (23) & (25), and (b) follows from a standard 
identity. 

To prove the upper bound for delay, we hrst consider the case of L hxed and show that 

n{N, L) < log 2 L 'i N. (29) 

We proceed by induction in L; the base case of L = 1 is immediate. For the inductive step, we have 

n{N,2L) 


(a) 


i-AT 


N 




log2 ^ . log2 L 


i=2 


(30) 


<1 + 2 ' 

i 1 + (1 - ^2-"^) log2L<log2 (2L) 

where (a) follows by the inductive assumption and (24) & (26), and (b) follows from a standard 
identity. 

We now consider the case of N hxed and show that 

n{N, L) < log 2 N+1 V L. (31) 

We proceed by induction in N] the base case of = 1 is immediate. For the inductive step, let 

g{i) = log2(2i) = log 2 i + 1. (32) 

Observe that the recurrence for Tb{-,-) can be written as 

n{N,2L) = l + E[n{Z,L)] (33) 

where Z is a random variable with PMF given as 

(^)2-^ *g{l,...,Ar-l} 


P(Z = i) = 


2-W-i) i = at 


and expected value 


E[Z] = ivQ + 2-^). 


(34) 


(35) 
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Applying the law of total expectation we have 

n{N,2L) = l + E[n{Z,L)\Z > 1]P(Z > 1) 
which follows from Tb{l,L) = 0. Snbstitnting the npper bonnd gives 

n{N, 2L) < 1 + E [g{Z)\Z > 1] (l - . 

We have 

^[g{Z)\Z>l] < g(E[Z\Z>l]) 

- ’ ( 2(1 -W 2 --V) ) = 

where (a) follows by Jensen’s ineqnality and (b) follows from 

E[Z] -E[Z|Z = 1]P(Z = 1) 

P(Z > 1) 

_ N (1 + 2-^) - N2-^ _ N 
~ 1 - N2-^ “ 2(1 - N2-^)' 

It follows that 

E[g{Z)\Z>l] {l-N2-^) <log^N 

and we conclnde Tb{N, 2L) < 1 + log 2 N. 

Binary search is an attractive qnantization strategy becanse it reqnires at most 2 bits per nser 
on average to compnte the argmax and the delay remains bonnded (by the nnmber of nsers) as 
the snpport set size grows. With the snpport set being halved at each ronnd of interaction, the 
snpport set size does not eqnal one nntil after log 2 L ronnds of interaction. Using binary search as 
a qnantization strategy for compnting the max has a delay of exactly log 2 L ronnds (c.f. Prop. 2). 

B. Max search 

As described at the end of the previons snbsection, binary search has a constant delay of log 2 L 
ronnds when the CEO wishes to compnte the max. We propose the following search strategy, which 
we refer to as max search: at each ronnd, the nsers indicate to the CEO whether their sonrce 
observation is the largest possible valne in the cnrrent snpport. If at least one nser replies in the 
affirmative, the interaction stops and the max has been fonnd. Only if none of the nsers have 
the largest valne does the interaction continne. This strategy is motivated by the following two 
observations: 1) for a hxed snpport set size, as the nnmber of nsers increases, the PMF of the max 
becomes more and more peaked abont the larger snpport set valnes (cf. Lemma 1 & Fig. 4), and; 



(36) 

(37) 

(38) 

(39) 

(40) 
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Fig. 4. Probability mass function for maxi Xi when XiS are i.i.d. uniform (L = 16) as a function of the number of 
users. As N increases, the probability becomes more concentrated around the larger values. 


2 ) at every round of interaction, there is a quantization bin of size one and therefore a non-zero 
probability of the interaction ending in the current round. 

The rate for max search can be written recursively as 


Rm{N, L) = Nh{pL) + (1 - PLfRmiN, L-1) 


where pL = 1/T; the delay can be written recursively as 


Tm{N, L) = 1 + (1 - pL^TmiN, L-1). 


(41) 


(42) 


Proposition 6. We have 

Rm{N,L) = Nj2i^-j h{l/i) 
^ / -i 


It follows that 


lim 

N^OO 


Rm{N,L) 

N 



lim 

N^OO 


Tm{N,L) = 1. 


(43a) 

(43b) 


(44) 


Proof: By induction on L. The base case of L = 2 is immediate. For the inductive step we have 


Rm{N,L) 


= Nh{l/L) + 


L-1 

L 


N L-1 

i=l 


L-1 


N 


h{l/i) 


(45) 


which follows from the inductive assumption and (41) & (43a). The proof for follows the same 
arguments. ■ 
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This scheme has very low rate and delay as the number of users N gets larger. Using this scheme, 
however, the CEO does not have the ability to select a desired rate-delay trade-off. In the next 
subsection, we extend this simple search strategy to a family of search strategies that give the CEO 
the ability to operate at a desired rate-delay trade-off. We conclude by noting that even though max 
search is designed to enable the CEO to interactively compute the max. Prop. 4 establish it as a 
quantization strategy for computing the argmax as well. 

C. Extended max seareh 

We extend the quantization strategy of the previous section into a family of quantization strategies. 
The previous strategy worked by asking the users to indicate whether or not their state W = max 
at each iteration t, terminating when at least one user replied in the affirmative. We extend this 
strategy by asking the users to indicate which of the K — \ largest values of X^^'> they have or 
indicating their state is not one of these values. For example: consider L = 5 and K = 4. The 
quantizer for this parameter set would be n = (2,1,1,1). Like the quantization strategy of the 
previous section, this family of strategies has the property that iteration continues if and only if all 
N users’ states are in the hrst bin. Unlike the previous strategy, we are not able to write a closed- 
form expression for the rate and delay components when selecting quantizers from this family of 
quantizers. However, we are able to prove several non-trivial and important properties of this family. 

We begin by giving a formal description of the family. For notational compactness, let Ik be the 
/c-tuple of all ones and denote tuple concatenation as 0. Suppose we are in a state s = {N, L). Dehne 

E,{L) = {{L — /c 0 1) 0 lfc_i : A; = 2,..., T} . (46) 

For example: C{4) = {(3,1), (2,1,1), (1,1,1,1)}. Our hrst result shows that for any quantizer n G 
£(T), permutation of the bin sizes results in a quantizer with a higher cost. Recall Assumption 1 of 
§IV where we took the search space of quantizers to be the set of partitions instead of compositions. 
Our justihcations for this assumption were concerns of computational complexity (cf. (17) & (18)) 
and the observation that the optimal solution was still found when searching over partitions for 
small problem instances (Fig. 2, right). Since permutation of an integer partition gives (in general) 
a composition, the next result is a further justihcation of Assumption 1. 

Proposition 7. For eomputing the max of the users’ values, the K-level quantizer n = (L — AT 0 1)0 
1k-i has lower eost than any other quantizer obtained by permutation. 


DRAFT 


September 8, 2015 



21 


Proof: Let be the quantizer obtained from n by shifting the bin of size L — K+1 m locations 
to the right. For example ns = I 3 0 (L — iL + 1) 0 1k-4- The stage cost of the quantizer 


(1 - X)R (a«, + Ar (a^, 


(47) 


is invariant to permutation. The expected cost to go of the quantizer n is 


{N,L-K + 1) 


(48) 


and the expected cost to go of the quantizer rim is 

N 

{i,L-K+i) 

i=l 


P{i) 


fN\ fL-K+iy 

vl; V L ) 


(49) 


Taking the difference we see 

N-l 

E/^«C'M^'^(bT-/L0l)>O (50) 

1=1 

and conclude that n has better cost than rim- As the choice of m was arbitrary, the result holds for 
any permutation. ■ 

An attractive property of C{L) is that |/1(L)| = L — 1 where as the number of all quantizers is 
exponential in L. If we could show that C{L) was sufficient for solving (4) (instead of the set of 
all quantizers) optimally, it would represent a signihcant reduction in computational complexity. As 
a hrst step, we show for a given quantizer n ^ ^(L) how to select a quantizer G >C(L) that 
asymptotically in N has performance no worse than n. 

Consider a /L-bin quantizer n = (rii,... ,nx)] if ni = L — K + then n G -C(L). Otherwise 
m < L — K + 1 and n ^ C,{L). Let Ur = {L — K + 1) ® £ ^(T) be the quantizer with the 

same number of bins as n. For example: if n = (2,2,1) then = (3,1,1). 

For notational compactness in the rest of the section, denote the current state as s = (A^, L) 
(which is hxed and known) and the state in the next iterations as s' = {N'^ L') (which are discrete 
random variables whose PMF depends on the select quantizer). The cost when using n is 


Cr^{N, L) = {1- X)NH{n) 0 A 0 [C{N', L')] 


(51) 


where 


k N 


[N' = i,L' = nj\C{i,nj), 

j=i 1=1 


(52a) 
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and 

[N' = z,L' = n,] = 

\ L J \ ’ 

and pj = The cost when nsing is 

Cr^^N, L) = {1- X)NH{n,) + A + [C{N', L')] 

where 

[C{N\ L')] = ^ j C{N, L-K+1). (54) 

Taking the difference we have 

Ar,,r.. = Cr,{N,L)-Cr^^{N,L) 

= {1 - X)N {H{n) - H{nr)) (55) 

+ E^ [C'(7V',L')]-E„, [^^(A^',^')]- 

The difference in the stage costs of qnantizer n and is expressed in terms of the difference in 
entropies of the indnced probability mass fnnctions H{n) — H{nj.) > 0 where the ineqnality follows 
from the fact that q is majorized by h and xlogx is convex [37]. 

Unfortnnately, the difference in the expected cost to go is not always positive. In fact, the difference 
in the expected cost to go can be negative enongh to offset the positive difference in the qnantizer 
rates. In Fig. 5 we consider the case of 77 = 2 and L = 16 and compare the costs of the non- 
£(16) qnantizer n = (11,5) to the rir G £(16) qnantizer (15,1). For certain valnes of A, Cn{N,L) 
is less than Cn^{N, L). This does not disprove the snfficiency of £(16) in solving (4) optimally; 
the range of A for which Cn{N,L) < Cn^{N,L) does not coincide with the range of A for which 
C{N, L) = Cn^{N, L). The above derivation allows ns to prove that £(£) is asymptotically snfficient 
for solving (4) optimally. 

Proposition 8. £(£) is asymptotically sufficient for minimizing the cost associated with interactively 
computing the max. The set C{L) cannot be made smaller without losing this property. For a given 
value of L and n ^ £(£) 

lim /\n,nr > 0 (56) 

N^OO 

where G £(£) has the same number of bins as n. 



(52b) 


(53) 
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Fig. 5. Comparison of the cost for n = (11,5) and rir = (15,1) for the state s = {N = 2,L = 16). rir does not 
outperform n in terms of cost for all A. 


Proof: The proof proceeds by induction with an inductive assumption that only quantizers from 
£(lW) for t > 1 are used for subsequent rounds. 

For the state {N,L), A^^nr is asymptotically (in N) non-negative. By assumption n ^ C-{L), 
therefore L —iC-|-l>ni>n 2 >---> iik > 1- This, in turn, implies that hi = {ni)®'^L-K+i-ni £ 
C{L — iC -h 1) is a valid quantizer for the state s' = — K + 1) and 


Cm {s') < (1 - X)N^'+^^H{hi) + A 


Next, there exists some k* such that n ,2 > • • • > rik* > 1; if not then n G P{L). We have that 


CM{i,nj)>X j<k* CM{i,nj) = 0 j>k*. 


(58) 


Finally, for notational compactness let u = ^ < 1. 
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We then have 


^n,nr > (1 - {H{n) - H{nr) - 


k* N 


= i, = n,\ C (z,. 

j=ii=i 


— V 


N 


qi 


L-K + l 


N 


C{N, rii) + A 


(b) 


(1 - X)N (^H{n) - H{nr) - X 


k* N 

+ EEP^ [wh+i) = i, = rij] C {i, Hj) 

j=2 i=l 
(c) 

> (1 - X)N{H{n) - H{nr) - v^H{hi)) 

k* N 

+ ^ ^ [wh+i) = i, A - I^^A 

j=2 i=l 

= (1 — A)W (^H{n) — H{nr) — H{fii)^ 


+ A 


^l:k* 

~17 


N 


1 - 


ni 


TV 


L-iC+l 

ni-.k* 


N' 


where 


(a) follows from (52), (54), (55), (57), and (58) 

(b) follows from 


Pr 


Arh+1) ^ i Lit+i) ^ 


qi 


0 , 

ifY 


% < n 


i = n 


(59) 


(60) 


and 

(c) follows from (58) 

ni-.k* = L — k + k* > K — k + 1 and therefore the right hand side of (59) has a non-negative limit 
because 


lim 
N^oo \ L 


ni-k* 


N 


1 - 


ni 


N 


L-K + l 


N' 


= 0 . 


(61) 


The set C{L) contains one and only one quantizer for each possible bin size. If this set were 
smaller, then for a given n the quantizer (which has the same number of bins as n) may not be 
in C'{L) C C{L). ■ 

In general, there exists values of N and L for which C{L) is not sufficient. Let Q{L) be the set of 
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Fig. 6. For iV = 2: (left) A(-) and A(-) for £ & Q \ £. A{C) > 0 & A(Q \ £) = 0 for L > 35. (right) A(r, A) 
L G {36,46,56,66,76}: (top) absolute error and (bottom) relative error. Max. relative error for L = 76 is ~ 0.33%. 


all quantizers for support set of size L. For U G Q, define 


A{U, A) = minC'(A, g) 

q€U 


min (^(A, q) 
qeQ 


( 62 ) 


A{U) = max , X) A(Z^) = m^n A(Z//, A). 

Here A{U, A) is understood as the A-dependent “gap to optimality” when using quantizers from U 
only instead of all quantizers Q; A{U) and A{U) are (respectively) the worst-case and best-case gap 
to optimality. From these dehnitions, we have 0 < A{U) < A{U). 

The top left side of Fig. 6 is a plot of A(£(L)) and A(£(£)) and the bottom left side is a plot 
of A{Q{L) \ C{L)) and A{Q{L) \ C{L)) as a function of L. When N = 2 and L < 34, A(£) = 0 
and A(Q \ £) >0. This means that for all A, the optimal quantizer for N = 2, L can be taken 
from L and there is no value of A for which the optimal quantizer can be taken from Q \ C. These 
two inequalities imply that L is necessary and sufficient for minimizing the cost. When N = 2 and 
35 < L < 76, from the hgure 0 = A(£) < A(£) and 0 = A(Q \ £) < A(Q \ £). This means that 
there exists a value of A such that the optimal quantizer n* 0 C. The right side of Fig. 6 plots the 
gap to optimality for C{L) as a function of A for representative values of L. The top right of the 
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figure plots the absolute magnitude of this gap, whereas the bottom right shows the magnitude of 
this gap as a relative percentage of the optimal value. The worst-case gap is growing in L (as Fig. 6 
shows as well), but the gap is still very small, at less than 0.35%, when L = 76. There is a range 
of A > 0.6 for which gap to optimality is identically zero—this range is where the delay component 
of cost is weighted more heavily than the rate component. A minimum delay of 1 can be achieved 
with the quantizer (1,..., 1) which is in C{L). Finally, for Li < L 2 the interval of A for which C{Li) 
is not sufficient is a subset of the interval for which £(^ 2 ) is not sufficient. 

The previous counterexample was for the case of A = 2 and we needed L > 35 for C{L) to no 
longer be necessary and sufficient for optimally solving (4). When N gets larger, the value of L at 
which C{L) is no longer necessary and sufficient gets larger as well. 

In summary, when computing the max interactively we know that C{L)\ 

1) is not an optimal search space for solving (4) in general (cf. (62) & Fig. 6); 

2) is asymptotically {L fixed, N increasing) sufficient for solving (4) (cf. Prop. 8); 

3) has linear growth (vs. exponential for all quantizers) (cf. (46)), and; 

4) incurs a small decrease in performance when it is not optimal (cf. Fig. 6). 

As we show in §VI, adding the binary-search quantizers to C{L) results in a simplihed search space 
for computing the arg max with little to no incurred penalty. Depending upon the system where this 
interactive quantization strategy is employed, the large reduction in computation costs may more 
than make up for the small increase in cost that occurs when using C{L) for selecting quantizers. 


VI. Results 

In this section, we investigate the rate-delay trade-offs, both for the optimized scalar quantizer 
scheme (§IV) and the proposed heuristics (§V). For brevity, when we refer to optimal rate-delay 
trade-offs, we are referring to the rate-delay trade-offs of the optimized scalar quantizer scheme. We 
begin by considering the optimal rate-delay trade-offs for a collection of representative distributions 
and show that the uniform distribution represents a worst-case distribution. This makes sense as 
the uniform distribution is entropy-maximizing for a given support set size. We then investigate 
the rate-delay trade-offs for binary and max search, demonstrating that these schemes can closely 
approximate the optimal rate-delay trade-off. 
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bx{x;L,p) 





max 




0 2 4 6 8 10 12 


Delay [no. of rounds] 


Fig. 7. Rate-delay trade-offs for gxix; L,p) (top row) and bx{x; L,p) (bottom row) for various values of p. PMFs 
are shown on the left, rate-delay trade-offs for argmax in the center, and max on the right. 


A. Optimized interactive quantization rate-delay trade-offs 

For a comparison of the rate-delay trade-offs for various distributions, we consider the following 
representative distributions parameterized with L and p which effects the concentration of the 
distribution. 

f 1 — 

9 x{x]L,p) =x = 0, (63) 

and, 

6x(x;L,p) = x = 0, ...,L-1. (64) 

The distribution gx{x;L,p) is shown in Fig. 7 (top left); the effect of varying p is to vary the 
“distance” from a uniform distribution. Also plotted in Fig. 7 is the optimal rate-delay trade-offs for 
both the argmax (top center) and the max (top right) functions. As p is made smaller, the trade-offs 
get worse in that a larger delay is incurred for smaller rates. The distribution bx{x; L,p) is shown 
in Fig. 7 (bottom left); unlike gx{x] L,p) the “spread” of the distribution is not sensitive to the 
parameter p. Looking at the rate-delay trade-offs (bottom center & left), p has little effect on the 
performance of the optimized scheme. 
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iV = 4 





Fig. 8. Comparison of rate-delay trade-offs: (top, left) for uniform, gx{x; L,p), and bx{x; L,p) (L = 16), (top, right) 
when computing the max for varying number of users {N), and (bottom) when computing argmax vs. max for N = 2 
(right) and N = 4. The source distribution was uniform with support set size L = 16. 


Fig. 8 (top left) shows the rate-delay trade-offs for these distributions for both argmax and max as 
computed by solving (4) and finding the optimal homogeneous quantizer at each round. The trade¬ 
offs for uniform are worse than for the other distributions. For a given upper limit on delay, uniformly 
distributed sources will require more rate than the other two distributions. For a hxed alphabet size, 
as the number of users is increased (upper right), the trade-offs for the uniform distribution gets 
better. Fig. 8 (bottom row) shows how the rate-delay trade-offs for computing argmax and max 
become identical as the number of users increases (cf. Prop. 4). We see that when N is small, the 
CEO is able to compute the argmax with either a lower rate (fixed delay) or lower delay (fixed 
rate) than it would require for computing the max with the same fixed rate or delay. This difference 
is especially large in the low rate/high delay regime. Doubling the number of users from 2 to 4 
signihcantly reduces this difference. 
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Delay [no. of rounds] 


max 



Delay [no. of rounds] 


Fig. 9. Comparison of rate-delay trade-off for various quantizer search spaces when computing argmax (left) and 
max (right). The source distribution was uniform with support set size L = 16 and the number of users was Af = 4. 


B. Extended max seareh rate-delay trade-offs 

As noted in §V, as the size of the support set L increases, the number of possible quantizers 
gets large quickly. Based on the simple quantization strategies of binary search and max search, 
we proposed the extended max search family C{L) of quantizers. Fig. 9 shows the rate-delay trade¬ 
off when the search space of (4) is taken to be 1) binary & max search with the entropy coding 
quantizer (1,..., 1) (solid line); 2) binary & extended max search (which includes the entropy coding 
quantizer) (circle markers), and; 3) all possible quantizers (dashed line). The left side is for the case 
of computing the argmax and the right side is for the case of computing the max. Binary & max 
search together can achieve the minimum rate and minimum delay ends of the trade-off curve for both 
functions, but performs poorly in efficiently trading off delay for rate. For computing the argmax, 
extended max search is almost equal to the optimal trade-off curve, deviating in the low rate/high 
delay regime. For computing the max, extended max search equals the optimal trade-off curve. By 
Prop. 4 and Fig. 8, we know that as N increases, the cost of computing the argmax is equal to the 
cost of computing the max. Even though extended max search is designed with computation of the 
max in mind. Fig. 9 and Prop. 4 show that it is an effective quantizer search space for computing 
the argmax. 


VIE Conclusion 

In this paper, we considered the problem of a CEO computing a function of distributed users’ 
state as a model for distributed resource allocation. We proposed interactive scalar quantization as an 
achievable scheme for reducing the required rate to enable the CEO to compute the desired function 
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losslessly at the expense of an increase in delay. We solved for optimal rate-delay trade-off of scalar 
qnantization via a dynamic program. We established that asymptotically (in the nnmber of nsers 
N), the cost to compnte argmax is the same as the cost to compnte max. By considering simple 
qnantization schemes based on binary search and max search, we designed a family of qnantization 
strategies that is nearly optimal with a signihcantly rednced compntational cost. 

In the present work, we assnmed that every nser was nsing the same qnantizer (i.e. homogeneous 
quantization). We know that in the rate-distortion problem, heterogeneous quantization can achieve 
a lower rate for a given distortion than homogeneons qnantization [20], [21]. A fntnre direction for 
the present work wonld be to extend the model to incorporate different qnantizers at the different 
nsers. Small nnmerical experiments have demonstrated that snbstantial farther redaction in the rate 
reqnired to calcnlate the extremnm at a bonnded expected delay can be obtained by switching from 
homogeneons to heterogeneons designs. A potential obstacle is the dramatic increase in the size of 
the search space; the size of the search space is eqnal to the size of the search space of homogeneons 
qnantization raised to N. 

The cnrrent work demonstrates that the reqnired rate can be rednced by tolerating a small increase 
in delay; in a similar manner, the reqnired rate can be rednced by tolerating a small increase in 
distortion [20], [21]. The model of these two lines of inqniry conld be combined into a single framework 
to qnantify the rate savings that conld be realized by tolerating both delay and distortion. This wonld 
reqnire snitably modifying the cost fnnction of (4) to inclnde a term for distortion. At each ronnd, 
the CEO wonld decide if the distortion is low enongh to stop or if commnnication shonld continne. 
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